Image processing device and image processing method for color correction, and image processing system including the same

ABSTRACT

An image processing device includes a memory and a color correction circuit. The memory stores first correction information that used for correcting first pixel values among a plurality of pixel values. The plurality of pixel values are received from an auto-focus image sensor including first pixels configured to detect a phase difference and second pixels configured to detect an image. The first pixel values are obtained from the first pixels and correspond to a first color. The first correction information is used for correcting the first pixel values to correspond to a second color different from the first color. The color correction circuit receives first image frame data including the plurality of pixel values from the auto-focus image sensor, loads the first correction information from the memory, and generates first corrected image frame data by correcting the first pixel values included in the first image frame data to correspond to the second color based on the first correction information.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority under 35 USC § 119 to Korean PatentApplication No. 10-2020-0119077, filed on Sep. 16, 2020 in the KoreanIntellectual Property Office (KIPO), the contents of which are hereinincorporated by reference in their entirety.

BACKGROUND 1. Technical Field

Example embodiments relate generally to semiconductor integratedcircuits, and more particularly to image processing devices and imageprocessing methods for color correction, and image processing systemsincluding the image processing devices.

2. Description of the Related Art

A complementary metal oxide semiconductor (CMOS) image sensor is animage capturing device manufactured using CMOS processes. The CMOS imagesensor typically has lower manufacturing costs and a smaller pixel sizethan conventional charge coupled device (CCD) image sensors thattypically utilize relatively high-voltage analog circuits. Accordingly,CMOS-type image sensors may have lower power consumption requirementsrelative to CCD-type image sensors. Moreover, with the continuedimprovement in performance of CMOS image sensors, CMOS image sensors arebecoming more widely used in mobile electronic devices such as smartphones, tablet personal computers (PCs), and digital cameras.

In a digital image processing device such as a camera, it may be helpfulto detect a focus control state of a photographing lens to automaticallycontrol a focus of the lens. To achieve this, a conventional digitalimage processing device includes an additional focus detecting devicethat is separate and/or different from an image sensor. In this case,costs of the focus detecting device and/or an additional optical lensmay be increased and/or an entire size of the digital image processingdevice may be increased by the focus detecting device. To address theseconcerns, an auto-focus image sensor using a method of detecting a phasedifference has been developed. In addition, various schemes ofprocessing image signals output from the auto-focus image sensor havebeen researched for the image quality enhancement or improvement.

SUMMARY

At least one example embodiment of the present disclosure provides animage processing device capable of correcting pixel values of a specificcolor output from an auto-focus image sensor or an image sensor.

At least one example embodiment of the present disclosure provides animage processing method capable of correcting pixel values of a specificcolor output from an auto-focus image sensor or an image sensor.

At least one example embodiment of the present disclosure provides animage processing system including the image processing device

According to example embodiments, an image processing device includes amemory and a color correction circuit. The memory stores firstcorrection information for correcting first pixel values among aplurality of pixel values. The plurality of pixel values are receivedfrom an auto-focus image sensor including first pixels configured todetect a phase difference and second pixels configured to detect animage. The first pixel values are obtained from the first pixels andcorrespond to a first color. The first correction information is usedfor correcting the first pixel values to correspond to a second colordifferent from the first color. The color correction circuit receivesfirst image frame data including the plurality of pixel values from theauto-focus image sensor, loads the first correction information from thememory, and generates first corrected image frame data by correcting thefirst pixel values included in the first image frame data to correspondto the second color based on the first correction information.

According to example embodiments, in an image processing method, firstcorrection information for correcting first pixel values among aplurality of pixel values is generated. The plurality of pixel valuesare received from an auto-focus image sensor including first pixelsconfigured to detect a phase difference and second pixels configured todetect an image. The first pixel values are obtained from the firstpixels and correspond to a first color. The first correction informationis used for correcting the first pixel values to correspond to a secondcolor different from the first color. The first correction informationis stored in a memory. First image frame data including the plurality ofpixel values is received from the auto-focus image sensor. The firstcorrection information is loaded from the memory. First corrected imageframe data is generated by correcting the first pixel values included inthe first image frame data to correspond to the second color based onthe first correction information.

According to example embodiments, an image processing system includes anauto-focus image sensor and an image processing device. The auto-focusimage sensor includes first pixels configured to detect a phasedifference and second pixels configured to detect an image. The imageprocessing device performs an image processing operation on image framedata provided from the auto-focus image sensor. The image processingdevice includes a calibration circuit, a memory and a color correctioncircuit. The calibration circuit receives reference image frame dataobtained by capturing a reference pattern by the auto-focus imagesensor, compares original image data corresponding to the referencepattern with the reference image frame data, and calculates firstcorrection information based on a result of comparing the original imagedata with the reference image frame data. The first correctioninformation is used for correcting first pixel values among a pluralityof pixel values received from the auto-focus image sensor. The firstpixel values are obtained from the first pixels and correspond to afirst color. The first correction information is used for correcting thefirst pixel values to correspond to a second color different from thefirst color. The memory receives the first correction information fromthe calibration circuit, and stores the first correction information.The color correction circuit receives first image frame data includingthe plurality of pixel values from the auto-focus image sensor, loadsthe first correction information from the memory, and generates firstcorrected image frame data by correcting the first pixel values includedin the first image frame data to correspond to the second color based onthe first correction information. The first correction informationincludes a plurality of gains, a plurality of offsets and a plurality ofposition data. The plurality of gains and the plurality of offsets areused for converting the first pixel values corresponding to the firstcolor into first corrected pixel values corresponding to the secondcolor. The plurality of position data represent positions of the firstpixels. The calibration circuit and the memory generate the firstcorrection information and store the first correction information beforethe auto-focus image sensor normally operates.

According to example embodiments, an image processing device includes amemory and a color correction circuit. The memory stores firstcorrection information for correcting first pixel values among aplurality of pixel values. The plurality of pixel values are receivedfrom an image sensor including a plurality of pixels configured todetect an image. The first pixel values correspond to a first color. Thefirst correction information is used for correcting the first pixelvalues to correspond to a second color different from the first color.The color correction circuit receives first image frame data includingthe plurality of pixel values from the image sensor, loads the firstcorrection information from the memory, and generates first correctedimage frame data by correcting the first pixel values included in thefirst image frame data to correspond to the second color based on thefirst correction information.

In the image processing device, the image processing method, and theimage processing system according to example embodiments, the pixelvalues of the phase detection pixels included in the auto-focus imagesensor may be corrected or converted into heterogeneous colors. Theimage processing device may obtain the correction information for thecorrection operation in advance, may store the correction information inthe memory in advance, may load the correction information for eachimage frame, and may perform the correction operation based on theloaded correction information, instead of generating and calculating thecorrection information for the correction operation for each imageframe. In addition, the pixel values received from the image sensorother than the auto-focus image sensor may also be corrected orconverted into heterogeneous colors based on the correction informationobtained and stored in advance. Accordingly, the amount of calculation,calculation time, power consumption, etc. for the correction operationmay be reduced, and the image processing device may have improved orenhanced performance.

BRIEF DESCRIPTION OF THE DRAWINGS

Illustrative, non-limiting example embodiments will be more clearlyunderstood from the following detailed description taken in conjunctionwith the accompanying drawings.

FIG. 1 is a block diagram illustrating an image processing deviceaccording to example embodiments.

FIG. 2 is a block diagram illustrating an image processing systemaccording to example embodiments.

FIG. 3 is a block diagram illustrating an example of an auto-focus imagesensor included in an image processing system according to exampleembodiments.

FIG. 4 is a circuit diagram illustrating an example of a pixel includedin a pixel array included in an auto-focus image sensor of FIG. 3.

FIGS. 5A, 5B and 5C are diagrams illustrating an example of a pixelarray included in an auto-focus image sensor of FIG. 3.

FIGS. 6A, 6B and 6C are diagrams for describing an example of performingauto-focus using an auto-focus image sensor of FIG. 3.

FIGS. 7A, 7B, 7C, 8A, 8B and 9 are diagrams for describing an operationof an image processing device according to example embodiments.

FIG. 10 is a block diagram illustrating an image processing deviceaccording to example embodiments.

FIG. 11 is a block diagram illustrating an example of a calibrationcircuit included in an image processing device of FIG. 10.

FIGS. 12 and 13 are block diagrams illustrating an image processingdevice according to example embodiments.

FIG. 14 is a block diagram illustrating an image processing deviceaccording to example embodiments.

FIG. 15 is a block diagram illustrating an image processing systemaccording to example embodiments.

FIG. 16 is a plan view of an example of a pixel array included in animage sensor included in an image processing system of FIG. 15.

FIGS. 17A, 17B, 17C, 17D, 18A, 18B, 18C and 18D are diagrams fordescribing an operation of an image processing device according toexample embodiments.

FIG. 19 is a flowchart illustrating an image processing method accordingto example embodiments.

FIG. 20 is a flowchart illustrating an example of generating firstcorrection information in FIG. 19.

FIGS. 21 and 22 are flowcharts illustrating an image processing methodaccording to example embodiments.

FIG. 23 is a flowchart illustrating an image processing method accordingto example embodiments.

FIG. 24 is a block diagram illustrating an electronic system includingan image processing device according to example embodiments.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Various example embodiments will be described more fully with referenceto the accompanying drawings, in which embodiments are shown. Thepresent disclosure may, however, be embodied in many different forms andshould not be construed as limited to the embodiments set forth herein.Like reference numerals refer to like elements throughout thisapplication.

FIG. 1 is a block diagram illustrating an image processing deviceaccording to example embodiments.

Referring to FIG. 1, an image processing device 100 includes a memory120 and a color correction circuit 140.

The memory 120 stores and outputs first correction information CINF1.The first correction information CINF1 is used for correcting firstpixel values among a plurality of pixel values that are received from anauto-focus image sensor (e.g., an auto-focus image sensor 220 in FIG. 2)located outside the image processing device 100. As will be describedwith reference to FIGS. 5A, 5B and 5C, the auto-focus image sensorincludes first pixels that detect a phase difference and second pixelsthat detect an image, the first pixel values are obtained from the firstpixels and correspond to a first color, and the first correctioninformation CINF1 is used for correcting the first pixel values tocorrespond to a second color different from the first color. Forexample, the first correction information CINF1 may include data forcorrecting or converting pixel values of phase detection pixels intoheterogeneous colors (or different colors). For example, each of theplurality of pixel values may include a grayscale value, a luminancevalue and/or a brightness value of a respective one of the plurality ofpixels.

In some example embodiments, the memory 120 may include any nonvolatilememories, e.g., an electrically erasable programmable read-only memory(EEPROM), a flash memory, a phase random access memory (PRAM), aresistive random access memory (RRAM), a nano floating gate memory(NFGM), a polymer random access memory (PoRAM), a magnetic random accessmemory (MRAM), a ferroelectric random access memory (FRAM), a thyristorrandom access memory (TRAM), etc., and/or any volatile memories, e.g., adynamic random access memory (DRAM), a static random access memory(SRAM), etc.

In some example embodiments, the first correction information CINF1 maybe pre-obtained (or obtained in advance) and may be pre-stored (orstored in advance) in the memory 120 before the image processing device100 normally operates and/or before the auto-focus image sensor normallyoperates.

In some example embodiments, the first correction information CINF1 maybe generated by a calibration circuit (e.g., a calibration circuit 110in FIG. 10) disposed inside or outside the image processing device 100.An operation of generating the first correction information CINF1 willbe described with reference to FIGS. 10 and 11.

The color correction circuit 140 receives first image frame data FDAT1including the plurality of pixel values from the auto-focus imagesensor, loads the first correction information CINF1 from the memory120, and generates first corrected image frame data CFDAT1 based on thefirst correction information CINF1 and the first image frame data FDAT1.For example, in one embodiment, the color correction circuit 140generates the first corrected image frame data CFDAT1 by correcting thefirst pixel values included in the first image frame data FDAT1 tocorrespond to the second color based on the first correction informationCINF1. For example, the first image frame data FDAT1 and the firstcorrected image frame data CFDAT1 may correspond to one image frameprovided from the auto-focus image sensor. An operation of the colorcorrection circuit 140 will be described in detail with reference toFIGS. 7 through 9.

The image processing device 100 according to example embodiments maycorrect or convert the pixel values of the phase detection pixelsincluded in the auto-focus image sensor into heterogeneous colors. Theimage processing device 100 may obtain the correction information CINF1for the correction operation in advance, may store the correctioninformation CINF1 in the memory 120 in advance, may load the correctioninformation CINF1 for each image frame, and may perform the correctionoperation based on the loaded correction information CINF1, instead ofgenerating and calculating the correction information for the correctionoperation for each image frame. Accordingly, the amount of calculation,calculation time, power consumption, etc. for the correction operationmay be reduced, and the image processing device 100 may have improved orenhanced performance.

FIG. 2 is a block diagram illustrating an image processing systemaccording to example embodiments.

Referring to FIG. 2, an image processing system 200 includes anauto-focus image sensor 220 and an image processing device 240.

The auto-focus image sensor 220 sequentially outputs a plurality ofimage frame data FDAT including information associated with an image ofan object based on incident light. Each of the plurality of image framedata FDAT may include a plurality of pixel values obtained from aplurality of pixels included in the auto-focus image sensor 220. Forexample, the first image frame data FDAT1 in FIG. 1 may be included inthe plurality of image frame data FDAT. A configuration and operation ofthe auto-focus image sensor 220 will be described in detail withreference to FIGS. 3 through 6.

The image processing device 240 sequentially receives the plurality ofimage frame data FDAT, and sequentially generates a plurality ofcorrected image frame data CFDAT by sequentially correcting theplurality of image frame data FDAT. The image processing device 240 maybe implemented according to example embodiments. The image processingdevice 240 may obtain and store in advance correction information (e.g.,the first correction information CINF1 in FIG. 1) used for correctionoperation, and may perform the correction operation for each image framebased on the correction information that is stored in advance.

In some example embodiments, the image processing device 240 may be theimage processing device 100 of FIG. 1. In other example embodiments, theimage processing device 240 may be one of image processing devices 100a, 100 b and 100 c of FIGS. 10, 12 and 13. The image processing device240 may be referred to as an image signal processor (ISP).

In some example embodiments, the image processing device 240 mayadditionally perform at least one image processing operation such asimage interpolation, demosaic, white balance, gamma correction, colorconversion, etc.

FIG. 3 is a block diagram illustrating an example of an auto-focus imagesensor included in an image processing system according to exampleembodiments.

Referring to FIG. 3, an auto-focus image sensor 500 includes a pixelarray 510, a correlated double sampling (CDS) block 530 and ananalog-to-digital converting (ADC) block 540. The auto-focus imagesensor 500 may further include a row driver 520, a digital signalprocessor 550, a ramp signal generator 560 and a timing controller 580.As is traditional in the field of the disclosed technology, features andembodiments are described, and illustrated in the drawings, in terms offunctional blocks, units and/or modules, or components described with˜er or ˜or. Those skilled in the art will appreciate that these blocks,units and/or modules are physically implemented by electronic (oroptical) circuits such as logic circuits, discrete components,microprocessors, hard-wired circuits, memory elements, wiringconnections, and the like, which may be formed using semiconductor-basedfabrication techniques or other manufacturing technologies. In the caseof the blocks, units and/or modules being implemented by microprocessorsor similar, they may be programmed using software (e.g., microcode) toperform various functions discussed herein and may optionally be drivenby firmware and/or software.

The pixel array 510 includes a plurality of pixels (or unit pixels) PXthat are arranged in a matrix formation. Each of the plurality of pixelsPX may be connected to a respective one of a plurality of rows RW1, RW2,. . . , RWX and a respective one of a plurality of columns CL1, CL2, . .. , CLY, where each of X and Y is a natural number greater than or equalto two. The pixel array 510 generates a plurality of analog pixelsignals VP1, VP2, . . . , VPY based on incident light. Pixelsimplemented in hardware, such as pixels PX may be referred to herein assensor pixels.

The plurality of pixels PX may include first pixels that detect a phasedifference and second pixels that detect an image. Configurations ofeach of the plurality of pixels PX and the pixel array 510 will bedescribed in detail with reference to FIGS. 4 and 5.

The row driver 520 may be connected to the plurality of rows RW1 throughRWX of the pixel array 510. The row driver 520 may generate drivingsignals to drive the plurality of rows RW1 through RWX. For example, therow driver 520 may drive the plurality of pixels PX included in thepixel array 510 row by row.

The correlated double sampling block 530 may include a plurality ofcorrelated double sampling circuits (CDSs) 530 a, 530 b, . . . , 530 c.The plurality of correlated double sampling circuits 530 a through 530 cmay be connected to the plurality of columns CL1 through CLY of thepixel array 510. The plurality of correlated double sampling circuits530 a through 530 c may perform a correlated double sampling operationon the plurality of analog pixel signals VP1 through VPY output from thepixel array 510.

The analog-to-digital converting block 540 includes a plurality ofanalog-to-digital converters 540 a, 540 b, . . . , 540 c. The pluralityof analog-to-digital converters 540 a through 540 c are connected to theplurality of columns CL1 through CLY of the pixel array 510 via theplurality of correlated double sampling circuits 530 a through 530 c.The plurality of analog-to-digital converters 540 a through 540 cperform a column analog-to-digital converting operation that convertsthe plurality of analog pixel signals VP1 through VPY (e.g., a pluralityof correlated double sampled analog pixel signals output from theplurality of correlated double sampling circuits 530 a through 530 c)into a plurality of digital signals CNT1, CNT2, . . . , CNTY in parallel(e.g., simultaneously or concurrently).

Each of the plurality of analog-to-digital converters 540 a through 540c may include a respective one of a plurality of comparators 542 a, 542b, . . . , 542 c and a respective one of a plurality of counters (CNTs)544 a, 544 b, . . . , 544 c. For example, the first analog-to-digitalconverter 540 a may include the first comparator 542 a and the firstcounter 544 a. The first comparator 542 a may compare the first analogpixel signal VP1 (e.g., the correlated double sampled first analog pixelsignal output from the first correlated double sampling circuit 530 a)with a ramp signal VRAMP to generate a first comparison signal CS1. Thefirst counter 544 a may count a level transition timing of the firstcomparison signal CS1 to generate the first digital signal CNT1.

Operations of the correlated double sampling block 530 and theanalog-to-digital converting block 540 may be performed on the pluralityof pixels PX included in the pixel array 510 row by row.

The plurality of correlated double sampling circuits 530 a through 530 cand the plurality of analog-to-digital converters 540 a through 540 cmay form a plurality of column driving circuits. For example, the firstcorrelated double sampling circuit 530 a and the first analog-to-digitalconverter 540 a may form a first column driving circuit.

The digital signal processor 550 may perform a digital signal processingoperation based on the plurality of digital signals CNT1 through CNTY.For example, an operation of obtaining auto-focus information using thefirst pixels and an operation of obtaining an image frame using thesecond pixels may be performed by the digital signal processor 550. Thedigital signal processor 550 may sequentially output a plurality ofimage frame data FDAT generated by the digital signal processingoperation.

In some example embodiments, the digital signal processor 550 may beomitted, and the above-described digital signal processing operation maybe performed by a signal processing device (e.g., the image processingdevice 240 in FIG. 2) located outside the auto-focus image sensor 500.

The ramp signal generator 560 may generate the ramp signal VRAMP. Thetiming controller 580 may control overall operation timings of theauto-focus image sensor 500, and may generate control signals includinga count enable signal CNT_EN, a clock signal (not illustrated), etc.

FIG. 4 is a circuit diagram illustrating an example of a pixel includedin a pixel array included in an auto-focus image sensor of FIG. 3.

Referring to FIG. 4, a pixel (or unit pixel) 600 may include aphotoelectric conversion unit 610 and a signal generation unit 612. Thefirst pixels AFP that detect a phase difference and the second pixelsthat detect an image among the plurality of pixels PX included in thepixel array 510 in FIG. 3 may have the same pixel structure and circuitstructure, except that configurations of micro lenses and color filtersare different from each other as will be described with reference toFIGS. 5A, 5B and 5C.

The photoelectric conversion unit 610 may perform a photoelectricconversion operation. For example, the photoelectric conversion unit 610may convert the incident light into photo-charges during an integrationmode. If an image sensor including the pixel 600 is a complementarymetal oxide semiconductor (CMOS) image sensor, image information on anobject to be captured may be obtained by collecting charge carriers(e.g., electron-hole pairs) in the photoelectric conversion unit 610proportional to intensity of the incident light through an open shutterof the CMOS image sensor during the integration mode.

The signal generation unit 612 may generate an electric signal (e.g., ananalog pixel signal VP) based on the photo-charges generated by thephotoelectric conversion operation during a readout mode. If the imagesensor including the pixel 600 is the CMOS image sensor, the shutter maybe closed, and the analog pixel signal VP may be generated based on theimage information in a form of the charge carriers during the readoutmode after the integration mode. As illustrated in FIG. 4, in someembodiments, the pixel 600 may have four-transistor structure includingfour transistors.

For example, the signal generation unit 612 may include a transfertransistor 620, a reset transistor 640, a driving transistor 650, aselection transistor 660 and a floating diffusion node 630. The transfertransistor 620 may be connected between the photoelectric conversionunit 610 and the floating diffusion node 630, and may include a gateelectrode receiving a transfer signal TX. The reset transistor 640 maybe connected between a power supply voltage VDD and the floatingdiffusion node 630, and may include a gate electrode receiving a resetsignal RX. The driving transistor 650 may be connected between the powersupply voltage VDD and the selection transistor 660, and may include agate electrode connected to the floating diffusion node 630. Theselection transistor 660 may be connected between the driving transistor650 and an output terminal outputting the analog pixel signal VP, andmay include a gate electrode receiving a selection signal SEL.

In some example embodiments, one signal generation unit may be shared bya plurality of photoelectric conversion units, to form a single unitpixel having a plurality of photoelectric conversion units.

FIGS. 5A, 5B and 5C are diagrams illustrating an example of a pixelarray included in an auto-focus image sensor of FIG. 3.

FIG. 5A is a plan view of an example of a pixel array included in anauto-focus image sensor. FIG. 5B is a cross-sectional view of an exampleof pixels included in a pixel array included in an auto-focus imagesensor. FIG. 5C is a plan view of an example of a color filter arrayincluded in the pixel array of FIG. 5A.

Referring to FIG. 5A, a pixel array 512 included in an auto-focus imagesensor may include a plurality of pixels PX11, PX12, PX13, PX14, PX15,PX16, PX21, PX22, PX23, PX24, PX25, PX26, PX31, PX32, PX33, PX35, PX36,PX41, PX42, PX43, PX44, PX45, PX46, PX51, PX52, PX53, PX54, PX55, PX56,PX61, PX62, PX63, PX64, PX65 and PX66. The pixel array may be repeatedin this manner hundreds, thousands, or more times to be arranged in alarger array that forms the image sensor.

As described above, the plurality of pixels PX11 through PX16, PX21through PX26, PX31 through PX33, PX35, PX36, PX41 through PX46, PX51through PX56, and PX61 through PX66 may include first pixels that detecta phase difference and second pixels that detect an image. The firstpixels may be referred to as auto-focus (AF) pixels, and the secondpixels may be referred to as normal pixels, or image pixels. In thepixel array 512 according to example embodiments, the AF pixels forobtaining AF information and the normal pixels may be disposed in onepixel array 512. In an example of FIG. 5A, the pixel PX33 may be the AFpixel, and the remaining pixels PX11 through PX16, PX21 through PX26,PX31, PX32, PX35, PX36, PX41 through PX46, PX51 through PX56, and PX61through PX66 may be normal pixels.

Micro lenses ML may be disposed on the plurality of pixels PX11 throughPX16, PX21 through PX26, PX31 through PX33, PX35, PX36, PX41 throughPX46, PX51 through PX56, and PX61 through PX66. A size of the micro lensof the AF pixel (e.g., the pixel PX33) may be larger than a size of themicro lens of the normal pixel (e.g., the pixel PX35).

Although FIG. 5A illustrates an example where six pixels are disposed inthe pixel array 512 in a first direction DR1, six pixels are disposed inthe pixel array 512 in a second direction DR2 crossing (e.g.,substantially perpendicular to) the first direction DR1, and the pixelarray 512 includes one AF pixel and thirty four normal pixels, exampleembodiments are not limited thereto. The size of the pixel array and thenumber of the pixels may be changed according to example embodiments.

Referring to FIG. 5B, a first pixel (or AF pixel) AFPX may include afirst photoelectric conversion region PD1, a second photoelectricconversion region PD2, a first color filter CF1 and a first micro lensML1. For example, the first pixel AFPX may correspond to the pixel PX33in FIG. 5A.

The first and second photoelectric conversion regions PD1 and PD2 may beformed in a substrate 511. The first color filter CF1 may be formed onthe first and second photoelectric conversion regions PD1 and PD2 (e.g.,along a third direction DR3 crossing (e.g., substantially perpendicularto) the first and second directions DR1 and DR2), and may be shared bythe first and second photoelectric conversion regions PD1 and PD2. Insome example embodiments, the first color filter CF1 may include twocolor filters that have the same color and formed on the first andsecond photoelectric conversion regions PD1 and PD2, respectively. Thefirst micro lens ML1 may be formed on the first color filter CF1, andmay be shared by the first and second photoelectric conversion regionsPD1 and PD2.

A second pixel (or normal pixel) NPX may include a third photoelectricconversion region PD3, a second color filter CF2 and a second micro lensML2. For example, the second pixel NPX may correspond to the pixel PX35in FIG. 5A.

The third photoelectric conversion region PD3 may be formed in thesubstrate 511. The second color filter CF2 may be formed on the thirdphotoelectric conversion region PD3. The second micro lens ML2 may beformed on the second color filter CF2.

In some example embodiments, the first, second and third photoelectricconversion regions PD1, PD2 and PD3 have the same size (e.g., the samearea in a plan view or on a plane). Thus, a size (e.g., an area in aplan view) of the first micro lens ML1 shared by the first and secondphotoelectric conversion regions PD1 and PD2 may be larger than a sizeof the second micro lens ML2. For example, the size of the first microlens ML1 may be about twice the size of the second micro lens ML2. Termssuch as “about” or “approximately” may reflect amounts, sizes,orientations, or layouts that vary only in a small relative manner,and/or in a way that does not significantly alter the operation,functionality, or structure of certain elements. For example, a rangefrom “about 0.1 to about 1” may encompass a range such as a 0%-5%deviation around 0.1 and a 0% to 5% deviation around 1, especially ifsuch deviation maintains the same effect as the listed range.

The photoelectric conversion regions PD1, PD2 and PD3 may correspond tothe photoelectric conversion unit 610 in FIG. 4, and may perform aphotoelectric conversion operation based on incident light to generatephoto-charges. For example, each of the photoelectric conversion regionsPD1, PD2 and PD3 may generate electron-hole pairs in response to theincident light, and may collect the electrons and/or the holes of theelectron-hole pairs. The photoelectric conversion regions PD1, PD2 andPD3 may include a photo diode, a photo transistor, a photo gate, apinned photo diode (PPD) and/or a combination thereof.

In some example embodiments, the color filters CF1 and CF2 may include ared filter, a green filter and/or a blue filter. In other exampleembodiments, the color filters CF1 and CF2 may include a yellow filter,a magenta filter and/or a cyan filter. The color filters CF1 and CF2 mayfurther include a white filter.

The micro lenses ML1 and ML2 may adjust a path of light entering themicro lenses ML1 and ML2 such that the light is focused on correspondingphotoelectric conversion regions PD1, PD2 and PD3.

In some example embodiments, the micro lenses ML1 and ML2 may form amicro lens array, and the color filters CF1 and CF2 may form a colorfilter array.

In some example embodiments, the pixel array may further include adevice isolation region 513 formed between the photoelectric conversionregions PD1, PD2 and PD3. The device isolation region 513 may formed ordisposed in the substrate 511, may extend substantially vertically froma surface of the substrate 511 in the third direction DR3 so as topenetrate the substrate 511 to isolate or separate the photoelectricconversion regions PD1, PD2 and PD3 from each other, and may include aninsulative material. The device isolation region 513 may have a meshstructure in a plan view.

In some example embodiments, the auto-focus image sensor may be afrontside illuminated image sensor (FIS) that operates in response tothe incident light passing through a front surface of the substrate 511or a backside illuminated image sensor (BIS) that operates in responseto incident light passing through the back surface of the substrate 511.Although not illustrated in detail, gate electrodes and wirings forforming the signal generation unit 612 in FIG. 4 may be disposed on thefront or back surface of the substrate 511.

Referring to FIG. 5C, a color filter array CFA1 included in the pixelarray 512 may include a red filter R, a green filter G, a blue filter Band a white filter W. In other words, the color filter array CFA1 mayhave an RGBW pattern.

The pixels PX32, PX36, PX41 and PX45 may include the red filter R. Thepixels PX12, PX16, PX21, PX25, PX33, PX43, PX52, PX56, PX61 and PX65 mayinclude the green filter G. The pixels PX14, PX23, PX54 and PX63 mayinclude the blue filter B. The pixels PX11, PX13, PX15, PX22, PX24,PX26, PX31, PX35, PX42, PX44, PX46, PX51, PX53, PX55, PX62, PX64 andPX66 may include the white filter W. As a result, in some embodiments,the AF pixels and the normal pixels included in the pixel array 512 mayhave the same color filter arrangement structure.

FIGS. 6A, 6B and 6C are diagrams for describing an example of performingauto-focus using an auto-focus image sensor of FIG. 3.

Referring to FIGS. 6A, 6B and 6C, light (or incident light) of an objectthat has passed through a lens 51 of an image capturing device (e.g., acamera) including an auto-focus image sensor passes through a micro lensarray 54 including a plurality of micro lenses (e.g., the micro lens ML1in FIG. 5B) so as to be introduced to a first AF pixel (R) 57 and asecond AF pixel (L) 58. The light inputted from a pupil 52 disposedabove a light axis 50 of the lens 51 is induced to the second AF pixel58, and the light inputted from a pupil 53 disposed under the light axis50 of the lens 51 is induced to the first AF pixel 57.

Continuous pixel outputs of the first and second AF pixels 57 and 58according to positions of the first and second AF pixels 57 and 58 areillustrated in FIGS. 6B and 6C. In each of FIGS. 6B and 6C, a horizontalaxis represents a position of each of the first and second AF pixels 57and, and a vertical axis represents an output value of each of the firstand second AF pixels 57 and. Referring to FIGS. 6B and 6C, a shape ofthe continuous output value of the first AF pixel 57 is substantiallythe same as that of the second AF pixel 58. However, positions (e.g.,phases) of the output values of the first and the second AF pixels 57and 58 may be different from each other as shown in FIG. 6B. This isbecause positions at which an image is formed based on the lightprovided from the pupils 52 and 53 of the lens 51 are different fromeach other. Thus, if the lens 51 is out of focus, the phases of theoutput values of the first and the second AF pixels 57 and 58 aredifferent as illustrated in FIG. 6B. If the lens 51 is in focus, theimage is formed at the same position as illustrated in FIG. 6C.

In addition, a direction of a focus difference may be determined basedon a difference between the phases of the output values of the first andthe second AF pixels 57 and 58. A front-focusing state means that thelens 51 focuses in front of the object. In the front-focusing state, thephase of the output value of the first AF pixel 57 is left-shifted froma phase of a focused state, and the phase of the output value of thesecond AF pixel 58 is right-shifted from the phase of the focused state.On the other hand, a back-focusing state means that the lens 51 focuseson a back of the object. In the back-focusing state, the phase of theoutput value of the first AF pixel 57 is right-shifted from the phase ofthe focused state, and the phase of the output value of the second AFpixel 58 is left-shifted from the phase of the focused state. Theshifted amount between the phases of the output values of the first andthe second AF pixels 57 and 58 may be used to obtain a deviation amountbetween focuses.

FIGS. 7A, 7B, 7C, 8A, 8B and 9 are diagrams for describing an operationof an image processing device according to example embodiments. FIGS.7A, 7B, 7C, 8A, 8B and 9 illustrate an operation when a pixel array ofan auto-focus image sensor has a structure illustrated in FIGS. 5A, 5Band 5C.

Referring to FIG. 7A, first image frame data FDAT11 that is providedfrom the auto-focus image sensor 500 and corresponds to one image framemay include a plurality of pixel values PV11W, PV12G, PV13W, PV14B,PV15W, PV16G, PV21G, PV22W, PV23B, PV24W, PV25G, PV26W, PV31W, PV32R,PV33G, PV34G, PV35W, PV36R, PV41R, PV42W, PV43G, PV44W, PV45R, PV46W,PV51W, PV52G, PV53W, PV54B, PV55W, PV56G, PV61G, PV62W, PV63B, PV64W,PV65G and PV66W that are obtained from the plurality of pixels PX11through PX16, PX21 through PX26, PX31 through PX33, PX35, PX36, PX41through PX46, PX51 through PX56, and PX61 through PX66.

The pixel values PV32R, PV36R, PV41R, PV45R that are obtained from thepixels PX32, PX36, PX41 and PX45 including the red filter R maycorrespond to a red color. The pixel values PV12G, PV16G, PV21G, PV25G,PV33G, PV34G, PV43G, PV52G, PV56G, PV61G and PV65G that are obtainedfrom the pixels PX12, PX16, PX21, PX25, PX33, PX43, PX52, PX56, PX61 andPX65 including the green filter G may correspond to a green color. Thepixel values PV14B, PV23B, PV54B and PV63B that are obtained from thepixels PX14, PX23, PX54 and PX63 including the blue filter B maycorrespond to a blue color. The pixel values PV11W, PV13W, PV15W, PV22W,PV24W, PV26W, PV31W, PV35W, PV42W, PV44W, PV46W, PV51W, PV53W, PV55W,PV62W, PV64W and PV66W that are obtained from the pixels PX11, PX13,PX15, PX22, PX24, PX26, PX31, PX35, PX42, PX44, PX46, PX51, PX53, PX55,PX62, PX64 and PX66 including the white filter W may correspond to awhite color. One pixel value may be generated from one photoelectricconversion region, and thus two pixel values PV33G and PV34G may beobtained from the pixel PX33, which is the AF pixel.

In some embodiments, the color filters disposed on the AF pixels are notused to form colors (e.g., to generate pixel values corresponding to aspecific color), and the color filters may be disposed on the AF pixelsonly for convenience of a process of manufacturing the color filterarray. Thus, the color filters disposed on the AF pixels may bedescribed as dummy color filters, and may not correspond to the colorpattern used for the remaining pixels of the pixel array. In addition,the color filter having a single color may be disposed on one AF pixelfor the accuracy of AF information. However, it may be useful for theoutput of the color filter array, for the purpose of processing imagepixels, to have the same color filter arrangement structure as when animage pixel array without AF pixels is used, for the accuracy of imageinformation. Thus it may be beneficial to correct the pixel valuesPV33G, PV34G, PV43G and PV44W to correspond to a WGGW pattern, e.g., tochange the pixel value PV33G corresponding to the green color tocorrespond to the white color, for image processing purposes other thanautofocusing.

Referring to FIG. 7B, first correction information CINF11 that is storedin the memory 120 and is used for correcting the first image frame dataFDAT11 may include correction data CI33 that is used for correcting thepixel value PV33G. For example, the pixel value PV33G may be correctedto allow that pixel to be used both for autofocusing and as an imagepixel.

In some example embodiments, the correction data CI33 may include a gain(or gain value) that is used for converting the pixel value PV33Gcorresponding to the green color into a corrected pixel value (e.g., acorrected pixel value PV33Wc in FIG. 7C) corresponding to the whitecolor, and position data that represents a position (or location) of thepixel PX33. Where a plurality of pixel data is to be corrected for aplurality of pixels, the correction data may include a plurality ofgains and a plurality of respective position data. For example,different pixels can have color corrected using the same or differentgains within the same image sensor.

For example, a following Equation 1 may be used for converting the pixelvalue PV33G corresponding to the green color into the corrected pixelvalue PV33Wc corresponding to the white color.

WV=α1·GV+β1·RV+γ1·BV  [Equation 1]

In the Equation 1, “WV” represents the corrected pixel value PV33Wc thatcorresponds to the white color and a result of the correction operation,“GV” represents the pixel value PV33G that corresponds to the greencolor and a target of the correction operation, “RV” represents a pixelvalue that is used for the correction operation, is obtained from apixel adjacent to the pixel PX33, and corresponds to the red color, and“BV” represents a pixel value that is used for the correction operation,is obtained from a pixel adjacent to the pixel PX33, and corresponds tothe blue color. Values α1, β1 and γ1 represent the gain, respectively.Therefore, a plurality of gains and a plurality of position data can beused to correct the pixel values.

When the correction operation is performed based on the Equation 1, notonly the pixel value PV33G obtained from the pixel PX33 but alsoadjacent pixel values obtained from adjacent pixels adjacent to thepixel PX33 may be used together to generate the corrected pixel valuePV33Wc. For example, the pixel value PV32R that is obtained from thepixel PX32 adjacent to the pixel PX33 and corresponds to the red colormay be used as the value “RV” in the Equation 1, and the pixel valuePV23B that is obtained from the pixel PX23 adjacent to the pixel PX33and corresponds to the blue color may be used as the value “BV” in theEquation 1. In some example embodiments, the pixel value PV34G that isobtained from the pixel PX34 adjacent to the pixel PX33 and correspondsto the green color and/or the pixel value PV43G that is obtained fromthe pixel PX43 adjacent to the pixel PX33 and corresponds to the greencolor may be additionally used as the adjacent pixel values.

For example, when a position of the pixel PX11 is defined as (1,1) and aposition of the pixel PX66 is defined as (6,6), the position data thatrepresents the position of the pixel PX33 may include a position valuecorresponding to (3,3), which is a position of a photoelectricconversion region generating the pixel value PV33G (e.g. the target ofthe correction operation) to be corrected among the pixel values PV33Gand PV34G obtained from the pixel PX33.

In other example embodiments, the correction data CI33 may include thegain and the position data, and may further include an offset (or offsetvalue) that is used for converting the pixel value PV33G correspondingto the green color into the corrected pixel value PV33Wc correspondingto the white color.

For example, a following Equation 2 may be used for converting the pixelvalue PV33G corresponding to the green color into the corrected pixelvalue PV33Wc corresponding to the white color.

WV=α2·GV+β2  [Equation 2]

In the Equation 2, “WV” represents the corrected pixel value PV33Wc thatcorresponds to the white color and a result of the correction operation,and “GV” represents the pixel value PV33G that corresponds to the greencolor and a target of the correction operation. A value α2 representsthe gain, and a value β2 represents the offset.

When the correction operation is performed based on the Equation 2, onlythe pixel value PV33G obtained from the pixel PX33 is used to generatethe corrected pixel value PV33Wc.

In still other example embodiments, the correction data CI33 may includethe gain, the offset and the position data, and may further include atleast one additional data for correcting the pixel value.

In FIG. 7B, a portion illustrated by a blank space, e.g., a portion inwhich correction data is not included or described, may be a regionwithout the correction data, e.g., a region in which the correctionoperation is not necessary or required or used. The first image framedata FDAT11 may include the pixel values for all pixels as illustratedin FIG. 7A, however, the first correction information CINF11 may includeonly the correction data CI33 for the pixel employing the correctionoperation. Thus, a size (or the amount of data) of the first correctioninformation CINF11 may be smaller than a size of the first image framedata FDAT11.

Referring to FIG. 7C, first corrected image frame data CFDAT11 may begenerated by correcting the first image frame data FDAT11 of FIG. 7Abased on the first correction information CINF11 of FIG. 7B. Forexample, the first corrected image frame data CFDAT11 may be generatedby correcting (or converting) the pixel value PV33G corresponding to thegreen color to the corrected pixel value PV33Wc corresponding to thewhite color based on the correction data CI33.

The remaining pixel values PV11W, PV12G, PV13W, PV14B, PV15W, PV16G,PV21G, PV22W, PV23B, PV24W, PV25G, PV26W, PV31W, PV32R, PV34G, PV35W,PV36R, PV41R, PV42W, PV43G, PV44W, PV45R, PV46W, PV51W, PV52G, PV53W,PV54B, PV55W, PV56G, PV61G, PV62W, PV63B, PV64W, PV65G and PV66Wincluded in the first corrected image frame data CFDAT11 may besubstantially the same as the pixel values PV11W, PV12G, PV13W, PV14B,PV15W, PV16G, PV21G, PV22W, PV23B, PV24W, PV25G, PV26W, PV31W, PV32R,PV34G, PV35W, PV36R, PV41R, PV42W, PV43G, PV44W, PV45R, PV46W, PV51W,PV52G, PV53W, PV54B, PV55W, PV56G, PV61G, PV62W, PV63B, PV64W, PV65G andPV66W included in the first image frame data FDAT11, respectively. Thepixel values shown in FIG. 7C may therefore represent data for an image(or portion of an image) sensed by the pixels of FIGS. 5A-5C.

Referring to FIG. 8A, in a conventional operation, a calibrationoperation CAL for generating correction information is performed basedon the first image frame data FDAT11 to generate calibrated first imageframe data FDAT11′, and the first corrected image frame data CFDAT11 isgenerated by performing a correction operation COR based on thecalibrated first image frame data FDAT11′. In this example, twooperations including the calibration operation CAL and the correctionoperation COR are performed, data corresponding to one image frame arecalculated and processed for each operation, and thus the amount ofcalculation, calculation time, power consumption, etc. are relativelylarge.

Referring to FIG. 8B, in an operation according to example embodiments,the first correction information CINF11 used for a correction operationCOR (e.g., as described for example in FIGS. 7A-7C) may be obtained andstored in advance, the stored first correction information CINF11 may beloaded when the first image frame data FDAT11 is received, and the firstcorrected image frame data CFDAT11 may be generated by performing thecorrection operation COR based on the loaded first correctioninformation CINF11. In this example, only one operation including thecorrection operation COR is performed, a relatively small size of thefirst correction information CINF11 is used during the correctionoperation COR, and thus the amount of calculation, calculation time,power consumption, etc. may be reduced.

Referring to FIG. 9, an example in which a plurality of corrected imageframe data CF1, CF2, CF3, CF4, CF5, CF6, CF7, CF8, CF9 and CF10 aresequentially generated when a plurality of image frame data F1, F2, F3,F4, F5, F6, F7, F8, F9 and F10 are sequentially received from theauto-focus image sensor 500 is illustrated.

In an example of FIG. 9, each of the plurality of image frame data F1through F10 may correspond to the first image frame data FDAT11 in FIGS.7A and 8B, the first correction information CINF11 may correspond to thefirst correction information CINF11 in FIGS. 7B and 8B, and each of theplurality of corrected image frame data CF1 through CF10 may correspondto the first corrected image frame data CFDAT11 in FIGS. 7C and 8B.

Similar to that described with reference to FIG. 8B, even when theplurality of image frame data F1 through F10 are sequentially received,the first correction information CINF11 stored in advance may be loaded,and the plurality of corrected image frame data CF1 through CF10 may besequentially generated by sequentially performing the correctionoperation COR based on the loaded first correction information CINF11.Thus, as compared to the conventional operation in which the calibrationoperation CAL and the correction operation COR are performed for eachframe (or every frame), the amount of calculation, calculation time,power consumption, etc. may be reduced.

Although FIG. 9 illustrates an example including ten image frame data F1through F10 and ten corrected image frame data CF1 through CF10, exampleembodiments are not limited thereto. For example, when first throughN-th image frame data are sequentially received from the auto-focusimage sensor 500, where N is a natural number greater than or equal totwo, first through N-th corrected image frame data may be sequentiallygenerated by sequentially correcting the first through N-th image framedata based on the first correction information CINF11.

Although example embodiments are described with reference to FIGS. 5A,5B, 5C, 6A, 6B, 6C, 7A, 7B, 7C, 8A, 8B and 9 based on a specificstructure of the pixel array, a specific structure of the pixels andspecific colors, example embodiments are not limited thereto. Forexample, example embodiments may be applied or employed when pixelvalues are generated from an AF pixel in which one micro lens and onecolor filter are shared by two or more photoelectric conversion regionsand when at least one pixel value corresponding to an arbitrary color iscorrected or converted to correspond to an arbitrary other color.

FIG. 10 is a block diagram illustrating an image processing deviceaccording to example embodiments. The descriptions repeated with FIG. 1will be omitted.

Referring to FIG. 10, an image processing device 100 a includes a memory120 and a color correction circuit 140. The image processing device 100a further includes a calibration circuit 110.

The image processing device 100 a may be substantially the same as theimage processing device 100 of FIG. 1, except that the image processingdevice 100 a further includes the calibration circuit 110.

The calibration circuit 110 may generate the first correctioninformation CINF1 that is stored in the memory 120. For example, thecalibration circuit 110 may generate the first correction informationCINF1 based on first original image data ORDAT1 that corresponds to areference pattern and first reference image frame data CRDAT1.

The reference pattern may represent a reference image for generating thefirst correction information CINF1 (e.g., the gain, the offset, etc.that are used for correcting the pixel value). For example, thereference pattern may be a uniform (or flat) image that corresponds toonly one grayscale value, such as a white image or a green image. Thereference pattern may include, for example, a specific pattern such as astripe. In some embodiments, the first original image data ORDAT1represents raw data of the reference pattern (e.g., data for a whiteimage, or data for a green image), and the first reference image framedata CRDAT1 represents a frame image obtained by capturing the referencepattern by the auto-focus image sensor.

In some example embodiments, the calibration circuit 110 may generateand store the first correction information CINF1 in the memory 120 inadvance before the auto-focus image sensor normally operates. Thus, thecalibration circuit 110 may be referred to as a pre-processing unit.

In some example embodiments, the calibration circuit 110 may be disabledor deactivated while the auto-focus image sensor normally operates. Forexample, the calibration circuit 110 may be enabled or activated at aninitial operation time of the image processing device 100 a or during aprocess of manufacturing the image processing device 100 a to generatethe first correction information CINF1, and then may be disabled ordeactivated and may not operate while the auto-focus image sensornormally operates. For example, the calibration circuit 110 may beenabled or disabled based on an enable signal EN.

FIG. 11 is a block diagram illustrating an example of a calibrationcircuit included in an image processing device of FIG. 10.

Referring to FIG. 11, a calibration circuit 110 a may include acomparator 112 and a calculator 114. Each of the comparator 112 andcalculator 114 may be a circuit, or may be implemented by computerprogram code executed by hardware.

The comparator 112 may receive the first original image data ORDAT1corresponding to the reference pattern and the first reference imageframe data CRDAT1 obtained by capturing the reference pattern by theauto-focus image sensor, and may compare the first original image dataORDAT1 with the first reference image frame data CRDAT1 to generatefirst comparison result data CR1.

The calculator 114 may calculate the first correction information CINF1based on the first comparison result data CR1, which is a result ofcomparing the first original image data ORDAT1 with the first referenceimage frame data CRDAT1. For example, the calculator 114 may calculate arelationship between the first color corresponding to a target of thecorrection operation and the second color corresponding to a result ofthe image capture operation. For example, the calculator 114 may obtainthe gain, the offset, etc. that are described with reference to theEquations 1 and 2 as a result of the calculation. For example, the firstoriginal image data ORDAT1 can correspond to a target color to achievefrom color correction (e.g., white), and the first reference image framedata CRDAT1 can correspond to an actual color (e.g., green) received byan image sensor (e.g., by the AF pixels) when the target color isapplied to the image sensor (e.g., the AF pixels). The gain, offset,etc., between that target color and actual color can be used later forcolor correction during operation.

FIGS. 12 and 13 are block diagrams illustrating an image processingdevice according to example embodiments. The descriptions repeated withFIG. 1 will be omitted.

Referring to FIG. 12, an image processing device 100 b includes a memory120 and a color correction circuit 140. The image processing device 100b may further include a selector 160. The selector 160 may be a circuit,or may be implemented by computer program code executed by hardware.

The image processing device 100 b may be substantially the same as theimage processing device 100 of FIG. 1, except that the memory 120further stores second correction information CINF2 and the imageprocessing device 100 b further includes the selector 160.

The memory 120 may store and output the second correction informationCINF2. As with the first correction information CINF1, the secondcorrection information CINF2 may be used for correcting the first pixelvalues corresponding to the first color among the plurality of pixelvalues that are received from the auto-focus image sensor, and may beused for correcting the first pixel values to correspond to the secondcolor different from the first color. In addition, the second correctioninformation CINF2 may have a value different from that of the firstcorrection information CINF1.

In some example embodiments, the first and second correction informationCINF1 and CINF2 may correspond to different color temperatures. Forexample, the first correction information CINF1 may correspond to afirst color temperature, and the second correction information CINF2 maycorrespond to a second color temperature different from the first colortemperature. However, example embodiments are not limited thereto, andthe first and second correction information CINF1 and CINF2 maycorrespond to the other different parameters associated with or relatedto colors.

The selector 160 may select one of the first correction informationCINF1 and the second correction information CINF2 based on the firstimage frame data FDAT1, and may output the selected correctioninformation SINF. The color correction circuit 140 may generate thefirst corrected image frame data CFDAT1 by correcting the first imageframe data FDAT1 based on the selected correction information SINF.

In some example embodiments, when the first and second correctioninformation CINF1 and CINF2 correspond to the different colortemperatures, the selector 160 may include a color temperaturedeterminer. For example, the color temperature determiner may determinea color temperature of the first image frame data FDAT1, may select thefirst correction information CINF1 when the color temperature of thefirst image frame data FDAT1 matches or corresponds to the first colortemperature, and may select the second correction information CINF2 whenthe color temperature of the first image frame data FDAT1 matches thesecond color temperature. For another example, when the colortemperature of the first image frame data FDAT1 does not match both thefirst and second color temperatures, the color temperature determinermay select one of the first correction information CINF1 and the secondcorrection information CINF2 that is closer to the color temperature ofthe first image frame data FDAT1.

Although FIG. 12 illustrates an example where the memory 120 stores twopieces of correction information CINF1 and CINF2 and the selector 160selects one of two pieces of correction information CINF1 and CINF2,example embodiments are not limited thereto. For example, the memory 120may store three or more correction information, and the selector 160 mayselect one of three or more pieces of correction information.

Referring to FIG. 13, an image processing device 100 c includes a memory120 and a color correction circuit 140. The image processing device 100c may further include a calibration circuit 110 and a selector 160.

The image processing device 100 c may be substantially the same as theimage processing device 100 of FIG. 1, except that the memory 120further stores the second correction information CINF2 and the imageprocessing device 100 c further includes the calibration circuit 110 andthe selector 160. The calibration circuit 110 may be similar to thecalibration circuit 110 in FIG. 10. The memory 120 and the selector 160may be substantially the same as the memory 120 and the selector 160 inFIG. 12, respectively.

The calibration circuit 110 may generate the first correctioninformation CINF1 based on the first original image data ORDAT1 and thefirst reference image frame data CRDAT1, and may generate the secondcorrection information CINF2 based on second original image data ORDAT2and second reference image frame data CRDAT2. For example, when thefirst correction information CINF1 corresponds to the first colortemperature, the first original image data ORDAT1 may represent thereference pattern corresponding to the first color temperature, and thefirst reference image frame data CRDAT1 may represent a frame imageobtained by capturing the reference pattern corresponding to the firstcolor temperature (or by capturing the reference pattern at the firstcolor temperature). When the second correction information CINF2corresponds to the second color temperature, the second original imagedata ORDAT2 may represent the reference pattern corresponding to thesecond color temperature, and the second reference image frame dataCRDAT2 may represent a frame image obtained by capturing the referencepattern corresponding to the second color temperature (or by capturingthe reference pattern at the second color temperature).

FIG. 14 is a block diagram illustrating an image processing deviceaccording to example embodiments. The descriptions repeated with FIG. 1will be omitted.

Referring to FIG. 14, an image processing device 300 includes a memory320 and a color correction circuit 340.

The image processing device 300 may be substantially the same as theimage processing device 100 of FIG. 1, except that the image processingdevice 300 performs a correction operation on an image frame receivedfrom an image sensor other than an auto-focus image sensor.

The memory 320 stores and outputs first correction information CINFA.The first correction information CINFA is used for correcting firstpixel values among a plurality of pixel values that are received from animage sensor (e.g., an image sensor 420 in FIG. 15) located outside theimage processing device 300. Unlike the auto-focus image sensor, theimage sensor includes only a plurality of pixels (e.g., normal pixels)that detect an image, the first pixel values are obtained from firstpixels among the plurality of pixels and correspond to a first color,and the first correction information CINFA is used for correcting thefirst pixel values to correspond to a second color different from thefirst color.

The color correction circuit 340 receives first image frame data FDATAincluding the plurality of pixel values from the image sensor, loads thefirst correction information CINFA from the memory 320, and generatesfirst corrected image frame data CFDATA based on the first correctioninformation CINFA and the first image frame data FDATA. For example, thecolor correction circuit 340 generates the first corrected image framedata CFDATA by correcting the first pixel values included in the firstimage frame data FDATA to correspond to the second color based on thefirst correction information CINFA.

The image processing device 300 according to example embodiments maycorrect or convert the pixel values of the pixels having a specificcolor and included in the image sensor into heterogeneous colors. Theimage processing device 300 may obtain the correction information CINFAfor the correction operation in advance, may store the correctioninformation CINFA in the memory 320 in advance, may load the correctioninformation CINFA for each image frame, and may perform the correctionoperation based on the loaded correction information CINFA, instead ofgenerating and calculating the correction information for the correctionoperation for each image frame. Accordingly, the amount of calculation,calculation time, power consumption, etc. for the correction operationmay be reduced, and the image processing device 300 may have improved orenhanced performance.

In some example embodiments, the image processing device 300 may furtherinclude a calibration circuit as described with reference to FIG. 10,may further include a selector as described with reference to FIG. 12,and/or may further include both the calibration circuit and the selectoras described with reference to FIG. 13. This correction operation can beperformed on a single pixel or group of pixels within a particularmatrix of pixels.

FIG. 15 is a block diagram illustrating an image processing systemaccording to example embodiments. The descriptions repeated with FIG. 1will be omitted.

Referring to FIG. 15, an image processing system 400 includes an imagesensor 420 and an image processing device 440.

The image sensor 420 sequentially outputs a plurality of image framedata FDAT including information associated with an image of an objectbased on incident light. The image sensor 420 may be implementedsimilarly to that described with reference to FIGS. 3 through 6, and astructure of the pixel array may be partially changed because the imagesensor 420 does not include AF pixels and includes only normal pixels.

The image processing device 440 sequentially receives the plurality ofimage frame data FDAT, and sequentially generates a plurality ofcorrected image frame data CFDAT by sequentially correcting theplurality of image frame data FDAT. The image processing device 440 maybe implemented according to example embodiments. For example, the imageprocessing device 440 may be the image processing device 300 of FIG. 14.

FIG. 16 is a plan view of an example of a pixel array included in animage sensor included in an image processing system of FIG. 15. Thedescriptions repeated with FIG. 5A will be omitted.

Referring to FIG. 16, a pixel array 514 included in an image sensor mayinclude a plurality of pixels PXA1, PXA2, PXA3, PXA4, PXA5, PXA6, PXB1,PXB2, PXB3, PXB4, PXB5, PXB6, PXC1, PXC2, PXC3, PXC4, PXC5, PXC6, PXD1,PXD2, PXD3, PXD4, PXD5, PXD6, PXE1, PXE2, PXE3, PXE4, PXE5, PXE6, PXF1,PXF2, PXF3, PXF4, PXF5 and PXF6.

All of the plurality of pixels PXA1 through PXA6, PXB1 through PXB6,PXC1 through PXC6, PXD1 through PXD6, PXE1 through PXE6, and PXF1through PXF6 may be normal pixels that detect an image, and micro lensesML may be disposed thereon. Each of the plurality of pixels PXA1 throughPXA6, PXB1 through PXB6, PXC1 through PXC6, PXD1 through PXD6, PXE1through PXE6, and PXF1 through PXF6 may be implemented as the secondpixel NPX in FIG. 5B.

FIGS. 17A, 17B, 17C, 17D, 18A, 18B, 18C and 18D are diagrams fordescribing an operation of an image processing device according toexample embodiments. The descriptions repeated with FIGS. 5C, 7A, 7B and7C will be omitted.

FIG. 17A is a plan view of an example of a color filter array includedin the pixel array 514 of FIG. 16. FIGS. 17B, 17C and 17D illustrate anoperation when a pixel array of an image sensor has a structureillustrated in FIGS. 16 and 17A. FIG. 18A is a plan view of anotherexample of a color filter array included in the pixel array 514 of FIG.16. FIGS. 18B, 18C and 18D illustrate an operation when a pixel array ofan image sensor has a structure illustrated in FIGS. 16 and 18A.

Referring to FIG. 17A, a color filter array CFAA1 included in the pixelarray 514 may include a red filter R, a green filter G, a blue filter Band a white filter W. The pixels PXA2, PXA4, PXA6, PXC2, PXC4, PXC6,PXE2, PXE4 and PXE6 may include the red filter R. The pixels PXA1, PXA3,PXA5, PXC1, PXC3, PXC5, PXE1, PXE3 and PXE5 may include the green filterG. The pixels PXB1, PXB3, PXB5, PXD1, PXD3, PXD5, PXF1, PXF3 and PXF5may include the blue filter B. The pixels PXB2, PXB4, PXB6, PXD2, PXD4,PXD6, PXF2, PXF4 and PXF6 may include the white filter W.

Referring to FIG. 17B, first image frame data FDATA1 that is providedfrom the image sensor 420 and corresponds to one image frame may includepixel values PVA2R, PVA4R, PVA6R, PVC2R, PVC4R, PVC6R, PVE2R, PVE4R andPVE6R that are obtained from the pixels PXA2, PXA4, PXA6, PXC2, PXC4,PXE2, PXE4 and PXE6 including the red filter R and correspond to a redcolor, pixel values PVA1G, PVA3G, PVA5G, PVC1G, PVC3G, PVC5G, PVE1G,PVE3G and PVE5G that are obtained from the pixels PXA1, PXA3, PXA5,PXC1, PXC3, PXC5, PXE1, PXE3 and PXE5 including the green filter G andcorrespond to a green color, pixel values PVB1B, PVB3B, PVB5B, PVD1B,PVD3B, PVD5B, PVF1B, PVF3B and PVF5B that are obtained from the pixelsPXB1, PXB3, PXB5, PXD1, PXD3, PXD5, PXF1, PXF3 and PXF5 including theblue filter B and correspond to a blue color, and pixel values PVB2W,PVB4W, PVB6W, PVD2W, PVD4W, PVD6W, PVF2W, PVF4W and PVF6W that areobtained from the pixels PXB2, PXB4, PXB6, PXD2, PXD4, PXD6, PXF2, PXF4and PXF6 including the white filter W and correspond to a white color.

Referring to FIG. 17C, first correction information CINFA1 that isstored in the memory 320 and is used for correcting the first imageframe data FDATA1 may include correction data CIB2, CIB4, CIB6, CID2,CID4, CID6, CIF2, CIF4 and CIF6 that are used for correcting the pixelvalues PVB2W, PVB4W, PVB6W, PVD2W, PVD4W, PVD6W, PVF2W, PVF4W and PVF6W.

Referring to FIG. 17D, first corrected image frame data CFDATA1 may begenerated by correcting the pixel values PVB2W, PVB4W, PVB6W, PVD2W,PVD4W, PVD6W, PVF2W, PVF4W and PVF6W that are included in the firstimage frame data FDATA1 of FIG. 17B and correspond to the white color tocorrected pixel values PVB2Gc, PVB4Gc, PVB6Gc, PVD2Gc, PVD4Gc, PVD6Gc,PVF2Gc, PVF4Gc and PVF6Gc corresponding to the green color, based on thefirst correction information CINFA1 of FIG. 17C. For example, an RGBWpattern may be converted into a Bayer pattern.

Referring to FIG. 18A, a color filter array CFAA2 included in the pixelarray 514 may include a red filter R, a green filter G, a blue filter Band a white filter W. The pixels PXC2, PXC6, PXD1 and PXD5 may includethe red filter R. The pixels PXA2, PXA6, PXB1, PXB5, PXC4, PXD3, PXE2,PXE6, PXF1 and PXF5 may include the green filter G. The pixels PXA4,PXB3, PXE4 and PXF3 may include the blue filter B. The pixels PXA1,PXA3, PXA5, PXB2, PXB4, PXB6, PXC1, PXC3, PXC5, PXD2, PXD4, PXD6, PXE1,PXE3, PXE5, PXF2, PXF4 and PXF6 may include the white filter W.

Referring to FIG. 18B, first image frame data FDATA2 that is providedfrom the image sensor 420 and corresponds to one image frame may includepixel values PVJ2R, PVJ6R, PVK1R and PVKSR that are obtained from thepixels PXC2, PXC6, PXD1 and PXD5 including the red filter R andcorrespond to a red color, pixel values PVG2G, PVG6G, PVH1G, PVHSG,PVJ4G, PVK3G, PVL2G, PVL6G, PVM1G and PVMSG that are obtained from thepixels PXA2, PXA6, PXB1, PXB5, PXC4, PXD3, PXE2, PXE6, PXF1 and PXF5including the green filter G and correspond to a green color, pixelvalues PVG4B, PVH3B, PVL4B and PVM3B that are obtained from the pixelsPXA4, PXB3, PXE4 and PXF3 including the blue filter B and correspond toa blue color, and pixel values PVG1W, PVG3W, PVGSW, PVH2W, PVH4W, PVH6W,PVJ1W, PVJ3W, PVJSW, PVK2W, PVK4W, PVK6W, PVL1W, PVL3W, PVLSW, PVM2W,PVM4W and PVM6W that are obtained from the pixels PXA1, PXA3, PXA5,PXB2, PXB4, PXB6, PXC1, PXC3, PXC5, PXD2, PXD4, PXD6, PXE1, PXE3, PXE5,PXF2, PXF4 and PXF6 including the white filter W and correspond to awhite color.

Referring to FIG. 18C, first correction information CINFA2 that isstored in the memory 320 and is used for correcting the first imageframe data FDATA2 may include correction data CIG1, CIG3, CIGS, CIH2,CIH4, CIH6, CM, CIJ3, CIJS, CIK2, CIK4, CIK6, CIL1, CIL3, CIL5, CIM2,CIM4 and CIM6 that are used for correcting the pixel values PVG1W,PVG3W, PVG5W, PVH2W, PVH4W, PVH6W, PVJ1W, PVJ3W, PVJ5W, PVK2W, PVK4W,PVK6W, PVL1W, PVL3W, PVL5W, PVM2W, PVM4W and PVM6W.

Referring to FIG. 18D, first corrected image frame data CFDATA2 may begenerated by correcting the pixel values PVJ1W, PVJ5W, PVK2W and PVK6Wthat are included in the first image frame data FDATA2 of FIG. 18B andcorrespond to the white color to corrected pixel values PVJ1Rc, PVJ5Rc,PVK2Rc and PVK6Rc corresponding to the red color, by correcting thepixel values PVG1W, PVG5W, PVH2W, PVH6W, PVJ3W, PVK4W, PVL1W, PVL5W,PVM2W and PVM6W that are included in the first image frame data FDATA2of FIG. 18B and correspond to the white color to corrected pixel valuesPVG1Gc, PVG5Gc, PVH2Gc, PVH6Gc, PVJ3Gc, PVK4Gc, PVL1Gc, PVL5Gc, PVM2Gcand PVM6Gc corresponding to the green color, and by correcting the pixelvalues PVG3W, PVH4W, PVL3W and PVM4W that are included in the firstimage frame data FDATA2 of FIG. 18B and correspond to the white color tocorrected pixel values PVG3Bc, PVH4Bc, PVL3Bc and PVM4Bc correspondingto the green color, based on the first correction information CINFA2 ofFIG. 18C. For example, an RGBW pattern may be converted into a Tetrapattern.

Although example embodiments are described with reference to FIGS. 16,17A, 17B, 17C, 17D, 18A, 18B, 18C and 18D based on a specific structureof the pixel array, a specific structure of the pixels and specificcolors, example embodiments are not limited thereto. For example,example embodiments may be applied or employed when the image sensorincludes a pixel array of an arbitrary structure and when at least onepixel value corresponding to an arbitrary color is corrected orconverted to correspond to an arbitrary another color.

FIG. 19 is a flowchart illustrating an image processing method accordingto example embodiments.

Referring to FIGS. 1, 2, 10 and 19, in an image processing methodaccording to example embodiments, the first correction information CINF1is generated (step S100). As described above, the first correctioninformation CINF1 is used for correcting the first pixel values amongthe plurality of pixel values that are received from the auto-focusimage sensor 220. The auto-focus image sensor 220 includes the firstpixels that detect the phase difference and the second pixels thatdetect the image, the first pixel values are obtained from the firstpixels and correspond to the first color, and the first correctioninformation CINF1 is used for correcting the first pixel values (e.g.,at least one of the pixel values) to correspond to the second colordifferent from the first color. Step S100 may be performed by thecalibration circuit 110 disposed inside or outside the image processingdevice.

The first correction information CINF1 is stored in the memory 120 (stepS200). The first image frame data FDAT1 including the plurality of pixelvalues is received from the auto-focus image sensor 220 (step S300). Thefirst correction information CINF1 is loaded from the memory 120 (stepS400). The first corrected image frame data CFDAT1 is generated bycorrecting the first pixel values included in the first image frame dataFDAT1 to correspond to the second color based on the first correctioninformation CINF1 (step S500). Steps S300, S400 and S500 may beperformed by the color correction circuit 140.

FIG. 20 is a flowchart illustrating an example of generating firstcorrection information in FIG. 19.

Referring to FIGS. 2, 10, 11, 19 and 20, when generating the firstcorrection information CINF1 (step S100), the first reference imageframe data CRDAT1 that is obtained by capturing the reference pattern bythe auto-focus image sensor 220 may be received (step S110). The firstreference image frame data CRDAT1 may be compared with the firstoriginal image data ORDAT1 corresponding to the reference pattern (stepS120). The first correction information CINF1 may be calculated based ona result of comparing the first original image data ORDAT1 with thefirst reference image frame data CRDAT1 (step S130). Steps S110 and S120may be performed by the comparator 112, and step S130 may be performedby the calculator 114.

FIGS. 21 and 22 are flowcharts illustrating an image processing methodaccording to example embodiments. The descriptions repeated with FIG. 19will be omitted.

Referring to FIGS. 1, 2 and 21, in an image processing method accordingto example embodiments, steps S100, S200 and S400 may be substantiallythe same as steps S100, S200 and S400 in FIG. 19, respectively.

The first through N-th image frame data are sequentially received fromthe auto-focus image sensor 220 (step S350). The first through N-thcorrected image frame data are sequentially generated by sequentiallycorrecting the first pixel values included in the first through N-thimage frame data to correspond to the second color based on the firstcorrection information CINF11 (step S550). The example of FIG. 21 mayalso be described as sequentially receiving the second through N-thimage frame data after step S300 in FIG. 19, and sequentially generatingthe second through N-th corrected image frame data by sequentiallycorrecting the second through N-th image frame data after step S500 inFIG. 19.

Referring to FIGS. 2, 12, 13 and 22, in an image processing methodaccording to example embodiments, the first correction information CINF1and the second correction information CINF2 are generated (step S150).The first and second correction information CINF1 and CINF2 are used forcorrecting the first pixel values corresponding to the first color tocorrespond to the second color, and are different from each other. Thefirst and second correction information CINF1 and CINF2 are stored inthe memory 120 (step S250). Steps S150 and S250 may be similar to stepsS100 and S200 in FIG. 19, respectively.

The first image frame data FDAT1 is received from the auto-focus imagesensor 220 (step S300). The first and second correction informationCINF1 and CINF2 are loaded from the memory 120 (step S450). One of thefirst and second correction information CINF1 and CINF2 is selectedbased on the first image frame data FDAT1 (step S600). The firstcorrected image frame data CFDAT1 is generated by correcting the firstpixel values included in first image frame data FDAT1 to correspond tothe second color based on the selected correction information SINF (stepS700). Steps S300, S450 and S700 may be similar to S300, S400 and S500in FIG. 19, respectively. Step S600 may be performed by the selector160.

FIG. 23 is a flowchart illustrating an image processing method accordingto example embodiments. The descriptions repeated with FIG. 19 will beomitted.

Referring to FIGS. 14, 15 and 23, in an image processing methodaccording to example embodiments, the first correction information CINFAis generated (step S1100). As described above, the first correctioninformation CINFA is used for correcting the first pixel values amongthe plurality of pixel values that are received from the image sensor420. The image sensor 420 includes only the plurality of pixels thatdetect the image, the first pixel values are obtained from the firstpixels among the plurality of pixels and correspond to the first color,and the first correction information CINFA is used for correcting thefirst pixel values to correspond to the second color different from thefirst color. Step S1100 may be performed by the calibration circuitdisposed inside or outside the image processing device.

The first correction information CINFA is stored in the memory 320 (stepS1200). The first image frame data FDATA including the plurality ofpixel values is received from the image sensor 420 (step S1300). Thefirst correction information CINFA is loaded from the memory 320 (stepS1400). The first corrected image frame data CFDATA is generated bycorrecting the first pixel values included in the first image frame dataFDATA to correspond to the second color based on the first correctioninformation CINFA (step S1500). Steps S1300, S1400 and S1500 may beperformed by the color correction circuit 340.

As will be appreciated by those skilled in the art, the inventiveconcept may be implemented using a system, method, computer programproduct, and/or a computer program product embodied in one or morecomputer readable medium(s) having computer readable program codeembodied thereon. The computer readable program code may be provided toa processor of a general purpose computer, special purpose computer, orother programmable data processing apparatus. The computer readablemedium may be a computer readable signal medium or a computer readablestorage medium. The computer readable storage medium may be any tangiblemedium that can contain or store a program for use by or in connectionwith an instruction execution system, apparatus, or device. For example,the computer readable medium may be a non-transitory computer readablemedium. The computer program product may interface with hardware toimplement functions such as those carried out by comparator 112 andcalculator 114 of FIG. 11, selector 160 of FIGS. 12 and 13, and othercomponents described herein.

FIG. 24 is a block diagram illustrating an electronic system includingan image processing device according to example embodiments.

Referring to FIG. 24, an electronic system 1000 may be implemented as adata processing device that uses or supports a mobile industry processorinterface (MIPI) interface. The electronic system 1000 may include anapplication processor 1110, an image sensor 1140, a display device 1150,etc. The electronic system 1000 may further include a radio frequency(RF) chip 1160, a global positioning system (GPS) 1120, a storage 1170,a microphone (MIC) 1180, a dynamic random access memory (DRAM) 1185 anda speaker 1190. In addition, the electronic system 1000 may performcommunications using an ultra wideband (UWB) 1210, a wireless local areanetwork (WLAN) 1220, a worldwide interoperability for microwave access(WIMAX) 1230, etc.

The application processor 1110 may be a controller or a processor thatcontrols operations of the image sensor 1140 and the display device1150.

The application processor 1110 may include a display serial interface(DSI) host 1111 that performs a serial communication with a DSI device1151 of the display device 1150, a camera serial interface (CSI) host1112 that performs a serial communication with a CSI device 1141 of theimage sensor 1140, a physical layer (PHY) 1113 that performs datacommunications with a PHY 1161 of the RF chip 1160 based on a MIPIDigRF, and a DigRF MASTER 1114 that controls the data communications ofthe physical layer 1161. A DigRF SLAVE 1162 of the RF chip 1160 may becontrolled through the DigRF MASTER 1114.

In some example embodiments, the DSI host 1111 may include a serializer(SER), and the DSI device 1151 may include a deserializer (DES). In someexample embodiments, the CSI host 1112 may include a deserializer (DES),and the CSI device 1141 may include a serializer (SER).

The CSI host 1112 may include the image processing device according toexample embodiments, and the CSI host 1112 and the image sensor 1140 mayform the image processing system according to example embodiments. Theimage sensor 1140 may be the auto-focus image sensor and/or the imagesensor included in the image processing system according to exampleembodiments. The CSI host 1112 and the image sensor 1140 may operatebased on the method according to example embodiments.

The inventive concept may be applied to various devices and systems thatinclude the image processing devices and the image sensors. For example,the inventive concept may be applied to systems such as a personalcomputer (PC), a server computer, a data center, a workstation, a mobilephone, a smart phone, a tablet computer, a laptop computer, a personaldigital assistant (PDA), a portable multimedia player (PMP), a digitalcamera, a portable game console, a music player, a camcorder, a videoplayer, a navigation device, a wearable device, an internet of things(IoT) device, an internet of everything (IoE) device, an e-book reader,a virtual reality (VR) device, an augmented reality (AR) device, arobotic device, a drone, etc.

The foregoing is illustrative of example embodiments and is not to beconstrued as limiting thereof. Although some example embodiments havebeen described, those skilled in the art will readily appreciate thatmany modifications are possible in the example embodiments withoutmaterially departing from the novel teachings and advantages of theexample embodiments. Accordingly, all such modifications are intended tobe included within the scope of the example embodiments as defined inthe claims. Therefore, it is to be understood that the foregoing isillustrative of various example embodiments and is not to be construedas limited to the specific example embodiments disclosed, and thatmodifications to the disclosed example embodiments, as well as otherexample embodiments, are intended to be included within the scope of theappended claims.

1. An image processing device comprising: a memory configured to storefirst correction information for correcting first pixel values among aplurality of pixel values, the plurality of pixel values being receivedfrom an auto-focus image sensor including first pixels configured todetect a phase difference and second pixels configured to detect animage, the first pixel values being obtained from the first pixels andcorresponding to a first color, the first correction information forcorrecting the first pixel values to correspond to a second colordifferent from the first color; and a color correction circuitconfigured to receive first image frame data including the plurality ofpixel values from the auto-focus image sensor, to load the firstcorrection information from the memory, and to generate first correctedimage frame data by correcting the first pixel values included in thefirst image frame data to correspond to the second color based on thefirst correction information.
 2. The image processing device of claim 1,further comprising: a calibration circuit configured to generate thefirst correction information stored in the memory.
 3. The imageprocessing device of claim 2, wherein the calibration circuit includes:a comparator configured to receive reference image frame data obtainedby capturing a reference pattern by the auto-focus image sensor, and tocompare original image data corresponding to the reference pattern withthe reference image frame data; and a calculator configured to calculatethe first correction information based on a result of comparing theoriginal image data with the reference image frame data.
 4. The imageprocessing device of claim 2, wherein the calibration circuit isconfigured to generate the first correction information and to store thefirst correction information in the memory before the auto-focus imagesensor normally operates.
 5. The image processing device of claim 4,wherein the calibration circuit is configured to be disabled while theauto-focus image sensor normally operates.
 6. The image processingdevice of claim 1, wherein the color correction circuit is configuredto: sequentially receive second image frame data through N-th imageframe data subsequent to the first image frame data from the auto-focusimage sensor, where N is a natural number greater than or equal to two,and sequentially generate second corrected image frame data through N-thcorrected image frame data by sequentially correcting the second imageframe data through the N-th image frame data based on the firstcorrection information.
 7. The image processing device of claim 1,wherein the first correction information includes: a plurality of gainsused for converting the first pixel values corresponding to the firstcolor into first corrected pixel values corresponding to the secondcolor; and a plurality of position data representing positions of thefirst pixels.
 8. The image processing device of claim 7, wherein thefirst pixel values and adjacent pixel values obtained from adjacentpixels adjacent to the first pixels among the second pixels are used forgenerating the first corrected pixel values.
 9. The image processingdevice of claim 7, wherein the first correction information furtherincludes: a plurality of offsets used for converting the first pixelvalues into the first corrected pixel values.
 10. The image processingdevice of claim 9, wherein the first pixel values are the only firstpixel values used for generating the first corrected pixel values. 11.The image processing device of claim 1, wherein: the memory isconfigured to further store second correction information different fromthe first correction information, the second correction information forcorrecting the first pixel values to correspond to the second color, andthe image processing device further includes: a selector configured toselect one of the first correction information and the second correctioninformation based on the first image frame data.
 12. The imageprocessing device of claim 11, wherein the color correction circuit isconfigured to generate the first corrected image frame data bycorrecting the first image frame data based on one of the firstcorrection information and the second correction information selected bythe selector.
 13. The image processing device of claim 11, wherein thefirst correction information corresponds to a first color temperature,and the second correction information corresponds to a second colortemperature different from the first color temperature.
 14. The imageprocessing device of claim 1, wherein each of the first pixels includes:a first photoelectric conversion region and a second photoelectricconversion region in a substrate; a first color filter on the first andsecond photoelectric conversion regions and having the first color; anda first micro lens on the first color filter and shared by the first andsecond photoelectric conversion regions.
 15. The image processing deviceof claim 14, wherein each of the second pixels includes: a thirdphotoelectric conversion region in the substrate; a second color filteron the third photoelectric conversion region; and a second micro lens onthe second color filter.
 16. The image processing device of claim 15,wherein: the first photoelectric conversion region, the secondphotoelectric conversion region and the third photoelectric conversionregion have a same size, and a size of the first micro lens is largerthan a size of the second micro lens.
 17. An image processing methodcomprising: generating first correction information for correcting firstpixel values among a plurality of pixel values, the plurality of pixelvalues being received from an auto-focus image sensor including firstpixels configured to detect a phase difference and second pixelsconfigured to detect an image, the first pixel values being obtainedfrom the first pixels and corresponding to a first color, the firstcorrection information being used for correcting the first pixel valuesto correspond to a second color different from the first color; storingthe first correction information in a memory; receiving first imageframe data including the plurality of pixel values from the auto-focusimage sensor; loading the first correction information from the memory;and generating first corrected image frame data by correcting the firstpixel values included in the first image frame data to correspond to thesecond color based on the first correction information.
 18. The imageprocessing method of claim 17, wherein generating the first correctioninformation includes: receiving reference image frame data obtained bycapturing a reference pattern by the auto-focus image sensor; comparingoriginal image data corresponding to the reference pattern with thereference image frame data; and calculating the first correctioninformation based on a result of comparing the original image data withthe reference image frame data.
 19. The image processing method of claim17, further comprising: sequentially receiving second image frame datathrough N-th image frame data subsequent to the first image frame datafrom the auto-focus image sensor, where N is a natural number greaterthan or equal to two; and sequentially generating second corrected imageframe data through N-th corrected image frame data by sequentiallycorrecting the second image frame data through the N-th image frame databased on the first correction information.
 20. (canceled)
 21. An imageprocessing system comprising: an auto-focus image sensor including firstpixels configured to detect a phase difference and second pixelsconfigured to detect an image; and an image processing device configuredto perform an image processing operation on image frame data providedfrom the auto-focus image sensor, wherein the image processing deviceincludes: a calibration circuit configured to receive reference imageframe data obtained by capturing a reference pattern by the auto-focusimage sensor, to compare original image data corresponding to thereference pattern with the reference image frame data, and to calculatefirst correction information based on a result of comparing the originalimage data with the reference image frame data, the first correctioninformation for correcting first pixel values among a plurality of pixelvalues received from the auto-focus image sensor, the first pixel valuesbeing obtained from the first pixels and corresponding to a first color,the first correction information for correcting the first pixel valuesto correspond to a second color different from the first color; a memoryconfigured to receive the first correction information from thecalibration circuit, and to store the first correction information; anda color correction circuit configured to receive first image frame dataincluding the plurality of pixel values from the auto-focus imagesensor, to load the first correction information from the memory, and togenerate first corrected image frame data by correcting the first pixelvalues included in the first image frame data to correspond to thesecond color based on the first correction information, wherein thefirst correction information includes: a plurality of gains and aplurality of offsets used for converting the first pixel valuescorresponding to the first color into first corrected pixel valuescorresponding to the second color; and a plurality of position datarepresenting positions of the first pixels, and wherein the calibrationcircuit and the memory are configured to generate the first correctioninformation and to store the first correction information before theauto-focus image sensor normally operates.
 22. (canceled)